ERIC ED500024: School Observation Measure Reliability Study
Publication date 2003-07
Topics ERIC Archive, Classroom Techniques, Federal Legislation, Observation, Educational Change, Measurement Techniques, Test Reliability, Teacher Education, Context Effect, Elementary Schools, Urban Schools, Rural Schools, Rural Urban Differences, Low Achievement, Teaching Methods, Computer Uses in Education, Sterbinsky, Allan, Ross, Steven M.
The No Child Left Behind (NCLB) stimulates school-wide reform by covering virtually all aspects of school operations including instruction, assessment, classroom management, professional development, parental involvement, school management, and curriculum. Since whole-school reforms typically change practices in the classroom, educators need a reliable classroom observation instrument that measures a wide variety of classroom practices. Historically, classroom observation instruments were designed for individual students, teachers, or classrooms, not the whole school. Additionally, the reliability of those instruments was shown to be context specific (e.g. length of observation, number of observations, timing in academic year). The School Observation Measure (SOM) is designed to measure a variety of classroom practices at the school level and the current reliability study was designed to incorporate many of the contextual variables elucidated by previous research. The current research study provides evidence of reliability for the SOM in a range of contexts. On average, the phi coefficient across all SOM items was 0.74 for five observations and 0.82 for eight observations at a school. Additionally, the percentage of variance attributable to observations was less than 1% while the percentage of variance attributable to differences between schools was 37% and the error term was 62% of five SOMs be conducted at a school to maintain adequate levels of reliability. Additional SOMs can be conducted at a school and tables are provided for educators to balance the cost of conducting additional SOMs with the asymptotic rate of return in increased reliability. Disaggregating reliability coefficients by elementary/high schools and rural/urban locations also evidenced reliability across contexts. The phi coefficients for five SOMs conducted at urban elementary schools averaged 0.72, compared to 0.76 for rural elementary schools. At high schools, the average phi coefficient associated with conducting only five SOMs was 0.60. Although reliabilities for high schools were somewhat lower than were those from elementary schools, this may be a statistical artifact due to the lower number of high schools in the sample compared to the number of elementary schools. It is recommended that a minimum of five SOMs be conducted at a school to maintain adequate levels of reliability. Additional SOMs can be conducted at a school and tables are provided for educators to balance the cost of conducting additional SOMs with the asymptotic rate of return in increased reliability. The SOM Instrument and Norms for SOM Items are appended. (Contains 8 tables.)
Ocr ABBYY FineReader 11.0
Uploaded by chris85 on